On Determining the Number of Clusters–a Comparative Study

In this paper, we perform one of the first empirical tests comparing several existing algorithms for determining the number of clusters in a data set (the gap statistic, X-means, G-means, data spectroscopic clustering and self-tuning spectral clustering). We use a large number of data sets randomly generated with varying distributions and parameters. The results show that the G-means and X-means perform best on the majority of test cases. We also explore ways to improve the gap statistic, and formulate the problem in the simplified continuous context to consider its theoretical basis.